Method and apparatus for monitoring an object

ABSTRACT

A method and system of monitoring an object (e.g., for change in configuration of a person) includes projecting a radiation pattern onto the object; recording at a first time first image data representing a portion of the projected radiation pattern on the object, the first image data representative of a three dimensional configuration of the object at the first time; recording at a second time second image data representing a portion of the projected pattern of radiation on the object, the second image data representative of a three dimensional configuration of the object at the second time; and processing the first and second image data to generate differential data representative of a change in the configuration of the object between the first and second times.

REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. Ser. No. 13/157,672 filedJun. 10, 2011, which is a continuation of PCT/EP2009/066790 Filed Dec.10, 2009.

FIELD OF THE INVENTION

The present invention relates to a method of monitoring an object and inparticular, but not limited to, monitoring a change in configuration ofa person.

BACKGROUND OF THE INVENTION

There are many applications where monitoring of a three dimensionalconfiguration of an object can be implemented, for example monitoring ofvital signs, including breathing and/or lung function of a person. Inaddition, monitoring of lung function can provide data which can be usedto assist in determining the presence of lung disease which is a globalhealth problem affecting millions of people. Effective medicalmanagement of lung disease requires assessment and monitoring of lungfunction. This is carried out largely by clinical observation or byspirometry (a technology developed in the 1930s).

There are restrictive limitations to present technologies for monitoringlung function, which fail to satisfy clinical needs; in particular,spirometry techniques require alert and co-operative patients to followinstructions whilst using an apparatus. These techniques are notsuitable for use on children under 5 years old, for critically ill orunconscious patients, or for the chronically sick or elderly. Suchpatients often cannot be assessed or monitored fully, leaving assessmentto subjective clinical observation. This leaves a subjective element toclinical decisions, for example a patient's transfer from intensive careto less resource-intensive general care or vice versa, resulting ininefficient use of resources and sub-optimal clinical management.

Other techniques for monitoring lung function require physical contactwith the patient, for example requiring sensors or reflective markers tobe placed on or attached to the subject. This may be medicallyinadvisable or impossible, for example in critical care situations suchas the monitoring of a burns victim, or the measurement of lung functionin premature babies.

It is an object of the invention to overcome such disadvantages.

SUMMARY OF THE INVENTION

In accordance with one aspect of the present invention, there isprovided a method of monitoring an object, the method including:

a) projecting a pattern of radiation onto an object for monitoring;

b) recording at a first instant in time first image data representing atleast a portion of said projected pattern of radiation on the object,said first image data being representative of a three dimensionalconfiguration of said object at said first instant in time;

c) recording at a second instant in time second image data representingat least a portion of said projected pattern of radiation on the object,said second image data being representative of a three dimensionalconfiguration of said object at said second instant in time; and

d) processing said first and second image data to generate differentialdata representative of a change in the configuration of said objectbetween the first and second instants in time.

The method of the invention advantageously provides for monitoringchanges of three dimensional configuration of an object accurately overa period of time. By using the projected pattern of radiation to obtaincorresponding image data for use in generating the differential data, itis not required for any contact with the object being monitored to bemade. Accordingly, the method of the present invention is simple toeffect, and is well suited for monitoring moving and/or delicateobjects, including objects in controlled or isolated environments, sinceno contact with the object is required for its monitoring; theinteraction with the object is in the form of the projected lightpattern on the object.

Since the differential data is based upon first and second image datawhich represents a three dimensional configuration of the object, andtherefore corresponds with the three dimensional nature of the objectbeing monitored, the change in configuration of the object is determinedaccurately using the method of the invention.

In a preferred embodiment of the present invention the method includesprocessing said first and second image data to generate differentialdata representative of a change in volume of said object between thefirst and second instants in time. The ability to determine a change involume of the object, over a period of time, without necessitatingcontact with the monitored object is advantageous and simple, andprovides volume data of the object which may be used in many variousapplications.

The portions in steps b) and c) may be substantially the same portion ofthe pattern. Alternatively, the portions in step b) and c) are differentportions of the pattern, allowing the method to generate thedifferential data for at least part of the object, whilst allowingmovement of the object.

In one embodiment of the present invention the method includesprocessing of the first and second image data to identify gross movementof the object over time between the first and second time instants.

Preferably, the pattern of radiation is a spatially-varying intensitypattern of non-ionising electromagnetic radiation which, wheninterpreted as an image, comprises a combination of distinct and readilylocatable elements, for example, combinations of spatial distributedcrossing points, corners or circular patches.

Preferably the electromagnetic radiation may be visible light ornon-visible infra-red radiation. Alternatively the electromagneticradiation may be of other frequencies.

Preferably, in step a) a sequence of different patterns of radiation isprojected onto the object, each of the different patterns beingprojected consecutively at a different instant in time.

In a preferred embodiment of the present invention the method includes:

recording further image data representing at least a portion of theprojected pattern of radiation on the object at a plurality of furtherinstants in time, said further image data being representative of athree dimensional configuration of said object at each of the pluralityof further instants in time;

processing said further image data to generate differential datarepresentative of a change in the configuration of said object betweenat least two of said plurality of further instants in time.

Recording and processing the further image data may be performed afterrecording and processing the first and second image data. The furtherimage data may include a plurality of image data recorded respectivelyat a plurality of different instants in time. Differential datagenerated using the further image data may therefore be used to monitorthe configuration change over a period of time of any duration, such asa prolonged period of time. This is advantageous, for example in medicalapplications of the invention such as continuous monitoring of thebreathing of a patient over time.

The further image data may be recorded at rates consistent with a givenapplication of the invention, dependent upon on the time resolutionrequired for monitoring changes in the three dimensional configurationof an object. In a preferred embodiment for a medical application, forexample monitoring of chest movement and breathing, this rate may be 50or 60 instants in time per second, i.e. 50 or 60 Hertz. Alternatively,the further image data may be recorded at a faster rate, for example 180or 360 instants per second, providing enhanced time resolution andspatial accuracy. Any recording rate may be selected so that the changein configuration of the object can be monitored as desired; for example,but not limited to a rate within the range of 50 to 360 Hertz. The ratemay be determined by the capabilities and/or settings of the apparatusperforming the monitoring, such as of the camera.

In a preferred embodiment the method includes approximating theconfiguration of a portion of the object upon which the pattern ofradiation is not projected. The first and/or second image data may beused for the approximating. Additionally, or alternatively, an algorithmmay be used to approximate the surface of the object upon which theintensity pattern is not projected. For example, a simple interpolationalgorithm for a flat surface may be used. Alternatively a more complexalgorithm may be used, to interpolate using a more complex surface, forexample, a part of a cylinder or sphere. The method may includegenerating the differential data using the approximated configuration ofthe portion of the object. Accordingly, the configuration of a nonscanned part of the object allows the change of configuration, forexample a volume, to be accurately determined without requiring contactwith the object by apparatus performing the invention method. The partof the object for which the configuration is approximated may be thereverse side of the object on which the radiation pattern is projectedupon. For example, if the pattern is projected upon a front side of aperson, the configuration of the rear side of the person may beapproximated.

In embodiments of the invention the object is a person or at least aportion of a person. In such embodiments the method may includeprocessing the first and second image data with bio-physical model datacharacterising the person. The bio-physical model data may includeparameters representative of any of size, age, sex and body type of theperson, and/or parameters representative of mechanical aspects of partsof the body. The parameters may relate to both normal and abnormalconditions of the person. This allows the method to be tuned to the bodytype of the person being monitored to yield enhanced accuracy ofdifferential data.

In preferred embodiments the change of configuration may be indicativeof a lung function of the person, such as breathing rate, dynamic airflow, and/or the dynamic change in breathing volume.

Patent application EP 1645841 describes a three dimensional shapemeasurement apparatus, but this does not record first and second imagedata according to the present invention, each being representative of athree dimensional configuration of an object being monitored. Therefore,it does not generate differential data using the first and second imagedata, as in the present invention. Moreover, the apparatus of the EP1645841 requires precise positioning of apparatus components, unlike theapparatus of the present invention.

In a preferred embodiment the method further includes calibrating ofapparatus arranged to perform at least steps a) and b), said calibratingincluding:

projecting a calibration pattern of radiation onto a calibration object;and

recording calibration image data representing at least a portion of saidprojected calibration pattern of radiation, said calibration image databeing representative of a three dimensional configuration of saidcalibration object, and

processing said first and second image data in step d) with thecalibration image data to generate the differential data.

Calibrating the apparatus advantageously ensures that the differentialdata generated is more accurate.

The method may include projecting a sequence of different calibrationpatterns of radiation onto the calibration object and recording thecalibration image data for each of said different calibration patternsof radiation.

In embodiments of the invention, the method may include performing saidcalibrating before performing steps a) to d) to determine a unit scalefor use in determining dimensions of the object from the first and/orsecond image data.

Preferably, the projecting in step a) further includes projecting saidcalibration pattern, or at least one of said calibration patterns ontothe object for monitoring, and the first and/or second image datarecorded in steps b) and/or c) includes calibration image datarepresentative of characteristics of a projection system and recordingsystem of apparatus for performing said method.

The projected pattern of radiation and the projected calibration patternor at least one of the calibration patterns of radiation may beinterleaved with each other. In this way calibrating may be performedsimultaneously as monitoring the object.

In accordance with a further aspect of the present invention there isprovided a method of monitoring an object, the method including:

a) projecting a pattern of radiation onto an object for monitoring;

b) recording at a first instant in time first image data representing atleast a portion of said projected pattern of radiation on the object,said first image data being representative a three dimensionalconfiguration of said object at said first instant in time;

c) recording at a second instant in time second image data representingat least a portion of said projected pattern of radiation on the object,said second image data being representative of a three dimensionalconfiguration of said object at said second instant in time; and

d) providing said first and second image data to a processing system forgenerating differential data representative of a change in theconfiguration of said object between the first and second instants intime.

In accordance with a yet further aspect of the present invention thereis provided a method of monitoring an object, the method including:

a) receiving first and second image data recorded by the method of:

i) projecting a pattern of radiation onto an object for monitoring;

ii) recording at a first instant in time the first image data whichrepresents at least a portion of said projected pattern of radiation onthe object, said first image data being representative of a threedimensional configuration of said object at said first instant in time;and

iii) recording at a second instant in time the second image datarepresenting at least a portion of said projected intensity pattern ofradiation on the object, said second image data being representative ofa three dimensional configuration of said object at said second instantin time, and

b) processing said first and second image data to generate differentialdata representative of a change in the configuration of said objectbetween the first and second instants in time.

Accordingly, steps a) to c) of the method of the invention may beperformed remotely from step d) of the invention, for example via acomputer network.

In accordance with another further aspect of the present invention thereis provided computer software arranged to perform the method accordingto the method of the present invention.

In accordance with yet a further aspect of the present invention thereis provided a data carrier storing the computer software of the presentinvention.

In accordance with a further aspect of the present invention there isprovided an apparatus for monitoring an object, the apparatuscomprising:

a projection system arranged to project a pattern of radiation onto anobject for monitoring;

a recording system arranged to record at first and second instants intime, respectively, first and second image data representing at least aportion of said projected pattern on the object, said first and secondimage data being representative of a three dimensional configuration ofsaid object at said first and second instants in time, respectively; and

a processing system arranged to process said first and second image datato generate differential data representative of a change in theconfiguration of said object between the first and second instants intime.

Preferably, the recording system is arranged to record a plurality ofimages of at least a portion of the projected pattern of radiation atone instant in time to record image data representative of a threedimensional configuration of the object, each of the plurality of imagesbeing recorded at a different viewpoint of said object.

In a preferred embodiment the apparatus comprises a pattern generatorarranged to generate said pattern of radiation for projection by theprojection system.

The pattern generator may be arranged to generate at least onecalibration pattern for projection onto the object for monitoring and/ora calibration object.

The projection system may be arranged to project a pattern of radiationof non-ionising electromagnetic radiation.

The projection system may be arranged to project the pattern ofradiation as visible radiation and the recording means is arranged todetect visible radiation.

The projection system may alternatively be arranged to project thepattern of radiation as infrared radiation and the recording system maybe arranged to detect infrared radiation.

Further, the pattern may be a spatially-varying intensity pattern, whichmay comprise a combination of spatially distributed crossing points,corners and/or circular patches.

Further features and advantages of the invention will become apparentfrom the following description of preferred embodiments of theinvention, given by way of example only, which is made with reference tothe accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a flow diagram giving an overview of the method of thepresent invention;

FIG. 2 shows a block diagram of a monitoring apparatus in accordancewith an embodiment of the present invention;

FIGS. 3 a and 3 b show a side view and an end view, respectively, of amonitoring apparatus in accordance with a first embodiment of thepresent invention;

FIG. 4 shows a side view of a monitoring apparatus in accordance with asecond embodiment of the present invention;

FIGS. 5 a and 5 b show monitoring apparatus incorporated as part of anincubator in accordance with a third embodiment of the presentinvention;

FIG. 6 shows a monitoring apparatus with a plurality of projectionsystems in accordance with a fourth embodiment of the present invention;

FIG. 7 shows a calibration pattern of radiation being projected onto aknown object;

FIGS. 8A and 8B show different patterns of radiation being projectedonto an object;

FIG. 9 shows a pattern of radiation projected onto a model of an infant;and

FIG. 10 shows an interpolated three-dimensional representation of themodel of the infant shown in FIG. 9 created in accordance with themethod of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The contents of U.S. patent application Ser. No. 13/157,672 filed Jun.10, 2011 and PCT/EP2009/066790 filed Dec. 10, 2009 are incorporated byreference in their entirety herein.

The present invention relates to a method of monitoring an object and anapparatus for performing the method. Further, the invention relates tocomputer software arranged to perform the method of the invention, andto a data carrier storing such software. In the exemplary embodimentsdisclosed hereinbelow reference is made to a method of monitoring aconfiguration indicative of a lung function of a person, by monitoring achange in volume of a thorax of a person. Such a method is anon-contact, non-invasive method that can safely monitor chest andabdomen dynamic volume changes and accurately derive air flow and lungfunction data to assist medical diagnosis and patient care. Lungfunction of a person includes, but is not limited to, thoracic volume,respiratory rate, tidal volume, inspiratory and expiratory times, forcedspirometry, regional changes in chest wall movement and regional lungfunction. Other applications of the invention are envisaged, asexplained below.

With reference to FIG. 1, which shows a flow diagram giving an overviewof the method of the present invention, a pattern of radiation isprojected onto an object that is to be monitored in step S1.

In step S2, first image data is recorded at a first instant in time. Thefirst image data is representative of at least a portion of the patternof radiation projected onto the object and contains data representativeof a three dimensional configuration of the object at the first instantin time.

In step S3, second image data is recorded at a second instant in time,different from the first instant in time. The second image data isrepresentative of at least a portion of the pattern of radiationprojected onto the object and contains data representative of a threedimensional configuration of the object at the second instant in time.

In step S4, the first and second image data is processed to generatedifferential data that is representative of a change in theconfiguration of the object between the first and second instants intime. The change in configuration may be a zero change if the objectconfiguration does not change between the first and second timeinstants.

Monitoring a configuration of an object in this manner has the advantageof being usable to provide an accurate computer generated representationof the object over time without the need of making physical contact withthe object. Further, the method of the invention allows monitoring of achange of a configuration of the object using the differential data,which may, for example, represent a change of surface shape, position,volume and/or a three dimensional shape of an object being monitored.Further, the method of the invention may be applied to monitor a changeof configuration of a part of an object, rather than of the wholeobject.

As will be explained further below, the method of the invention mayoptionally include: recording further image data representing at least aportion of the projected pattern of radiation on the object at aplurality of further instants in time, the further image data beingrepresentative of a three dimensional configuration of the object ateach of the plurality of further instants in time; processing thefurther image data to generate differential data representative of achange in the configuration of the object between at least two of theplurality of further instants in time.

FIG. 2 shows a block diagram of apparatus arranged to perform the methodof the invention, in accordance with an embodiment of the presentinvention. The apparatus 10 comprises a scanning system 12, dataprocessing and a man machine interface (MMI) system 14, a display system16, image data storage system 18, bio-physical modeling data 20 and astatistical database 22.

The scanning system 12 includes a recording system 24, including forexample at least two video cameras, at least one projection system 26,for example, a video projector, video capture hardware 28 forcontrolling the recording systems 24 and capturing an image from each ofthe recording systems 24, interface system 30 for connecting aprocessing platform, for example a personal computer (PC) to theprojection system 26, and a pattern generator 32 for creating at leastone pattern for the projection system 26.

In one embodiment of the present invention the monitoring apparatus 10may comprise two 200 Hz, 640×480 resolution cameras 24 and a videoprojector 26, all mounted on an adjustable mounting rig so as to definethe relative positioning of the cameras 24 and the video projector 26.Both the cameras 24 and the video projector 26 may be controlled by astandard personal computer (PC) platform incorporating a video capturecard for the cameras 24 and a graphics card to drive the video projector26. In other embodiments, the camera itself may at least partly processrecorded image data, for transmitting to a further processor, such as ofa standard PC, via a digital bus connection, such as Universal SerialBus (USB), Firewire™ or an Ethernet connection.

The apparatus of the present invention is advantageous in that theprecise geometric positioning and separation of the projection andrecording systems with respect to each other, and to the object beingmonitored, is flexible; i.e. precise positioning is not required. Thecalibration of the projecting and recording systems, as described later,allows the physical positioning of the projecting and recording systemsto be taken into account to generate accurate data.

Since each camera of the recording system may be arranged at a differentviewpoint of the object, and can record an image of the projectedpattern at a certain time instant, the combination of the imagesrecorded at that certain time instant by the different cameras allows agreater proportion of the object to be monitored. In this way, thecameras can be arranged to provide greater coverage of the object, forexample by positioning one camera to capture part of the object whichanother camera cannot accurately capture due to blind spots for example.Accordingly, the image data recorded for each time instant isrepresentative of a greater part of the object, and therefore allowsmore accurate data to be generated for the object between different timeinstants.

The data processing and MMI (man machine interface) system 14 comprisesfour main elements: three dimensional processing system 36, medical dataextraction system 38, presentation processing and system control system40, and calibration processing system 42.

The presentation processing and system control system 40 performsprocessing to allow control by a user of the monitoring apparatus 10,and may present results via the display system 16. Presentationprocessing and control is performed in conjunction with the image anddata storage processing system 18, by which data from the recordingsystem 24 recording images, image data from the 3D processing, derivedmedical data, and other intermediate computation results may be storedand retrieved via a storage device 44. The storage device 44 may be acomputer disk or solid-state storage device.

In performing the method of the present invention, the projection system26 projects, otherwise referred to herein as scanning, one or morepatterns of radiation onto an object for monitoring and each camera ofthe recording system 24 simultaneously captures, at differentviewpoints, an image representative of at least a portion of the atleast one projected pattern of radiation as it appears on the object. Inthis way, a plurality of images of the same and/or different portions ofthe projected pattern may be recorded at one instant in time. Thecaptured image from each camera of the recording systems 24 is thenprocessed and recorded as image data that is representative of a threedimensional configuration of the part of the object on which thecorresponding pattern portion was projected at that instant in time;such as for recording the first image data for the first time instant.The image data recorded at the second time instant, and for any furthertime instants, such as consecutive third and fourth time instants, maybe recorded in a similar manner by recording and processing a pluralityof images of the object at the particular time instant. The simultaneouscapture of an image by each camera of the recording system 24 at a timeinstant ensures data points between the image data for the differentimages can be accurately correlated during further data processing. Thecaptured images may be processed using known image processingtechniques, to reduce noise, for example.

The at least one pattern of radiation may include a spatially-varyingintensity pattern comprising a combination of distinct and readilylocatable pattern elements, for example a combination of spatiallydistributed crossing points, corners or small circular patches. Forexample, the at least one pattern of radiation projected on to theobject may comprise tessellated combinations of lines and points. Thepattern may include non-illuminated pattern patches, spaced in betweenilluminated patches of the pattern. The pattern elements appeardistorted when projected on the object, and this distortion allowsthree-dimensional configuration data of the part of the part of theobject upon which the pattern elements are projected, such as the firstor second image data, to be derived from the captured image data; thismethod of scanning an object is commonly referred to as a structuredlight technique. The pattern of radiation may be a simple repeatedpattern and/or may be selected with a complex repeating pattern offeatures such as spots, light/dark edges, for example, in order to avoidambiguity in the images of the pattern recorded by the cameras, and tocapture higher resolution images of the object. Ambiguity occurs when apart of the pattern falling on the object is distorted in a way so as toappear the same as if the object was not there. Such ambiguity makes itdifficult to determine accurately a configuration of the object. Inother embodiments, a sequence of different radiation patterns may beprojected onto the object, each of the different patterns beingprojected consecutively at a different instant in time. The projectionsystem 26 may project the pattern of radiation of certain selectedwavelengths, for example visible radiation or invisible infraredradiation onto the object, although it is envisaged that the patternradiation may be of other wavelengths of the spectrum of non-ionisingand electromagnetic radiation. Accordingly, the recording system 24 isadapted to capture the projected radiation of the appropriatewavelength(s). The projection system 26 requires a normal incoherentlight source, rather than coherent laser illumination; this enablesmedically-safer scanning of a subject/object, for example a child,compared to a laser-based projection method.

The three dimensional processing system 36 processes the images capturedby the recording system 24 and extracts image data representative of thethree dimensional configuration of the object which correspond toequivalent points on the surface of the object. This processing mayinclude image analysis to maximise useable contrast and dynamic rangewithin the captured images, image noise reduction, location of specificfeatures of the projected patterns of radiation, resolution ofambiguities in an expected location of specific pattern features due topattern distortion by the object surface, suppression of data errors,and/or identification of silhouettes to assist tracking of grossmovement of the object when monitoring the object over time. The imagedata is then interpolated to create continuous surface representationsof the object being monitored.

The recording system 24 is arranged in embodiments of the invention torecord the same portion of the pattern at each of the first and secondinstants in time. In this way, assuming the object remains stationary,the change in configuration of the same part of the object can bemonitored at both the first and second instants. In other embodiments,the recording system 24 may be arranged to record a different portion ofthe projected pattern at the first and second time instants. This mayfor example allow image data to be recorded for the same part of theobject being monitored, despite gross movement of the object part, andtherefore a different part of the pattern falling on the object partunder observation.

Tracking algorithms may be used for the image data processing to tracklarger (gross) movement of the object being monitored to improve thecalculation accuracy of the image data by helping the discrimination ofwanted motion for monitoring of the scanned surface(s) of the objectfrom the overall (gross) movement of the object. The algorithms may alsodetect when the object is not within the field of view of the recordingsystems 24 and if the object is moving rapidly, for example, where theobject is a person, the algorithm may be arranged such that coughing ofthe person may be considered as a gross movement.

In addition, the tracking algorithm may be used to determine when agross movement of the object occurs and to indicate this to an end user,ignore image data occurring at those instants in time when the grossmovement occurs or determine the new position of the object if, forexample the object has geometrically moved position by rotation ortranslation.

The image data representative of the three dimensional configuration ofthe object may be used to obtain data indicative of a volumecorresponding to the scanned object area by processing consecutive(sequential) image data produced by the three dimensional processingsystem 36 of the object. Differential data can be generated between thefirst and second image data taken at the respective first and secondinstants in time, which represents a change in configuration of thescanned object between the two instants in time. The differential datamay be generated by comparing the first image data with the second imagedata, to identify differences between the three-dimensionalconfiguration of the object at the first instant in time and thethree-dimensional configuration of the object at the second instant intime. The differential data is indicative of these identifieddifferences, and therefore represents a change in configuration of theobject between the first and second instants. The two instants in timemay be two consecutive points recorded in time; alternatively, the twoinstants may be any two instants recorded in a period of time, which maybe separated by other time instants for which image data is recorded.Therefore, it is possible to track dynamically any changes in theconfiguration of at least part of the object between any two timeinstants and over a period of time encompassing a plurality of timeinstants. In embodiments where the object is a person the change inconfiguration may be representative of a change in the volume of atleast part of the object, for example, a change of the volume of thechest and/or abdomen cavities.

In embodiments where the apparatus is to be used in a medicalenvironment, for example as shown in FIGS. 3, 4 and 5, referred tolater, the medical data extraction system 38 performs an algorithmiccomputation to derive a volume change for the body part being monitoredby the apparatus from the differential data.

To assist in determining a configuration change, such as a volume, of atleast part of the object, an approximation of the volume of the scannedpart of the object is calculated, by approximating the configuration ofportions of the object part which are not visible to any of therecording systems 24 and/or which are not illuminated with the projectedpattern of radiation. For example, if the pattern is projected upon afront side of a person's chest being monitored, the pattern does notfall upon the rear side of the person's chest, and the configuration ofthe rear side of the chest cannot be determined by recording image dataof the rear side. However, the configuration of the rear side of theperson's chest may be approximated to determine at least a crude, andmore preferably a refined, configuration of the rear side of the chest.This approximately determined configuration of the object part withoutthe projected pattern may be processed with the recorded image data, forexample the first and second image data, to generate more accuratedifferential data for the change in configuration of the person's chestover time. The approximation may be determined using an interpolationalgorithm for a flat surface. Alternatively, a more complex algorithmmay be used to interpolate the configuration of the part without theprojected pattern, for example using a shape more representative of theobject part in question, in this case a person's chest. Such arepresentative shape may be part of a cylinder or a sphere, and/or mayperhaps use bio-physical data, as described later, of the type ofperson, or indeed of the specific person being monitored, to refine theapproximation. Using this approximated configuration in combination withthe recorded image data derived from the image data captured for thescanned object part, a change in configuration, such as a volume of theobject part being monitored, can be determined more accurately,providing more reliable image data representing the three dimensionalconfiguration of the object, and differential data. This is importantespecially in applications of the method requiring a high level of dataaccuracy, for example in the medical field, when the generated data maybe used to diagnose medical conditions and appropriate treatments.Moreover, for delicate objects such as ill patients, which cannot beeasily moved, this approximation allows the patient to remain still, andlying on a bed for example, whilst still being able to determineaccurate image data of the three dimensional configuration of the objectand differential data.

Accordingly, processing of the data captured by the recording system 24is used to create dynamic three dimensional models of the subject's bodymovement, volume, surface area and other similar physical parameters.The breathing and respiration rate can then be calculated by comparingconcurrent data images to generate differential data that isrepresentative of a change in volume.

Information may then be displayed to an end user in a numerical and/orgraphical format via a display system, and/or may be transmitted viadata network interfaces provided as part of the processing platform(e.g. personal computer) to other systems, for remote display oradditional analysis. Output information can be categorised and used aspart of a decision support process or expert system linked to clinicalmanagement and care schemes. As the reconstruction may provide a fullthree dimensional representation of the object, in some embodimentsusing the approximation of the object part not illuminated with theradiation pattern, it is possible to view the object surface on-screenfrom any novel viewpoint.

Breathing airflow relating to lung function can be computed from dynamicvolume changes in the object's volume. This process may be augmented bycomparison with data in the statistical database 22, which contains datafrom prior reference (trial) operations of the apparatus and/or fromprior measurement operations of the apparatus on a number of differentpersons. In embodiments of the invention, the statistical database dataprovides standardised, statistically-averaged parameters for selectedsets of persons according to selected granges of age, sex, weight andbody types. Comparison using the statistical database data assistsenhanced interpretation of computed dynamic volume changes as comparedwith statistical norms. The statistical database data may also be usedto provide refinement of the scaling and reference parameters for thebio-physical model data 20.

The accuracy of volume calculations may be improved by the bio-physicalmodeling data 20, which integrates the image data recorded with avolumetric model of the object, using mapping and ‘best fit’ algorithmicprocessing, for example. The volumetric or bio-physical model data 20comprises a ‘physics-based’ approximation of the anatomical structure ofthe object or subject being scanned and models the physical propertiesof the object area, for example, the physical and elastic properties ofthe chest/abdomen, and movement constraints due to joint and skeletalconstruction. The bio-physical model data 20 is based on generic modelsfor the object under consideration with physical modeling parametersadjusted and calibrated according to average values according to subjecttype, age, sex, body mass and/or other configurations. The bio-physicalmodel data can include stored data from prior reference (trial)operations.

FIGS. 3 a and 3 b show exemplary parts of the monitoring apparatus inaccordance with an embodiment of the present invention. The apparatus 10comprises first and second cameras as part of the recording system 24,which may be, for example, Allied Vision Technology Pike F032B cameras,and the projection system 26, which may be, for example, an Acer P1625projector, attached to a mounting frame 50. The angle that each of therecording systems 24 is mounted with respect to the mounting frame 50may vary from about 10 degrees to about 40 degrees and depends on thedistance from the recording system 24 to the object 52. In thisembodiment it can be seen that the projection system 26 projects apattern of radiation onto the chest and abdominal area of a subject 52laying prone on a surface 54. The projection rate may be for examplearound 50 or 60 Hz, or 180 Hz or 360 Hz. The recording rate from therecording system 24 is consistent with the projection rate, for exampleif the projection rate is 180 Hz, the recording rate would also be 180images per second for each of the first and second recording systems.Other projection and recording rates of the first, second and furtherimage data is envisaged within, but not limited to, the range of 50 to360 Hz; therefore, the projection and recording rates may be slower orfaster than rates of 50 or 360 Hz. The first and second cameras of therecording system 24 may be orientated so that the field of view of oneof the recording system completely or at least partially overlaps thefield of view of the other recording system. The apparatus 10 may bearranged for monitoring and diagnosis of, for example, adults, children,premature infants, non-cooperative subjects or where any form of contactmethod is impractical, for example, premature infants, burns victims,critical care patients. The apparatus 10 may be arranged so that thesubject can be monitored in an upright standing position as illustratedin FIG. 4.

Referring now to FIG. 5 a, there is shown an embodiment of parts ofmonitoring apparatus 10 in accordance with the present inventionincorporated an incubator for a baby. The monitoring device 10 comprisesa projection system 26 arranged to project radiation onto an reflectiveoptics device 64, for example a mirror, which reflects the radiationemitted from the projection system onto a baby 62 located in theincubator 60, and two cameras of a recording system 24. The monitoringapparatus 10 is mounted in a sealed housing 66 and mounted to aninterior portion surface of the incubator 60 as shown in FIG. 5 a. Itwill be appreciated that the monitoring apparatus 10 may be arranged tobe located on an exterior surface of the incubator 60 as shown in FIG. 5b.

FIG. 6 shows a set up of monitoring apparatus for performing the methodof the invention, according to a further embodiment. A plurality ofpatterns of radiation may be projected onto the object simultaneously,so that a greater coverage of the object being monitored is achieved.Accordingly, as shown in FIG. 6, more than one pattern of radiation isprojected onto the object, using the three projectors, onto two oppositesides of the object and onto a surface between the two opposite sides.Two cameras, arranged between the three projectors, each simultaneouslyrecord image data of the projected patterns at each of differentinstants in time, for processing to derive differential data.

In use the apparatus of FIGS. 3, 4, 5 and 6 may be set-up in a specificconfiguration for the environment and/or object to be monitored. Theapparatus may be calibrated in order to perform the method of theinvention with an acceptable level of accuracy. The method of theinvention includes calibrating the apparatus for performing theinvention. The calibration includes projecting a calibration pattern ofradiation onto a calibration object; and recording calibration imagedata representing at least a portion of the projected calibrationpattern of radiation, the calibration image data being representative ofa three dimensional configuration of the calibration object. The firstand second image data in step d) may be processed with the calibrationimage data to generate the differential data based on the calibration.According to such calibration, in embodiments of the invention, twotypes of calibration may be conducted: projector and recording systemcalibration, and unit calibration.

Projector and recording system calibration may be performedautomatically and continuously during monitoring a change ofconfiguration of the object, described previously. This may be achievedby projecting one or more calibration patterns onto the object beingmonitored as part of projecting the radiation pattern onto the objectfor monitoring. The object being monitored may therefore be consideredto be a calibration object. The calibration pattern may be projected onthe calibration object, and the calibration image data recorded, at oneor a plurality of time instants different from the first and second timeinstants. For example, the calibration pattern may be projectedimmediately before the first time instant, and immediately before thesecond time instant, so that the calibration data is indicative of theprojector and recording system characteristics at the first and secondtime instants, without performing the calibration simultaneously asrecording the image data for monitoring the object. In this way thecalibration pattern(s) and the pattern for the object monitoring may beinterleaved with each other over time. The recorded first and/or secondimage data may then be processed using recorded calibration image datarepresentative of, in three-dimensional space, characteristics of theprojector system and parts of the recording system, for example of thecameras, such as their relative positions and optical focal points. Thisimproves the accuracy of the first and/or second image data.

By appropriate selection of the timing of projecting, and recording ofimage data of, the calibration pattern(s) and the pattern(s) for objectmonitoring, the projector and recording system calibration may beperformed rapidly such that it may be near-simultaneous with objectmonitoring. Accordingly, characteristics of the projector system andparts of the recording system, for example optical characteristicsand/or relative positioning of the projector and cameras, may berepeatedly determined during object monitoring, without needing to haltobject monitoring. In this way small changes over time in suchcharacteristics, for example due to a drift in focus, zoom factor and/orangular positioning of the projector and/or recording systems, may bedetermined by the projector and recording system calibration process andthereby used to correct the image data generated from monitoring theobject. Movement of the object being monitored does not affect thecalibration of the projector and recording system. Object movement canbe dealt with using tracking algorithms for example, described above.

Unit calibration is preferably conducted before monitoring of an objectaccording to the invention method, described with reference to FIG. 1,is begun. The unit calibration is used to determine absolutemeasurements accurately for when monitoring the object and involvesdetermining at least one absolute distance, so that during calibrationall the distances between the projector system, parts of the recordingsystem and the location where the object for being monitored will beplaced, such as a bed in the example that the object is a person, may beaccurately determined from the calibration image data and/or anydifferential data generated therefrom during calibration, for examplebetween calibration image data at different time instants. Once theapparatus is set up for the monitoring procedure, the unit calibrationis performed by projecting a calibration pattern of radiation, generatedby the projection system, onto a calibration object of known dimensions,and recording image data of the calibration object using the recordingsystem. Since the dimensions of the calibration object are known, thecalibration system 42 can determine the absolute positions of therecording and projection systems. Accordingly, the calibration system 42can determine a unit scale for dimensional units, against which imagedata recorded during monitoring of the invention method can be compared,so that absolute dimensions of the object being monitored can beaccurately determined. For example, a height and/or a width incentimeters of at least part of an object being monitored can beaccurately measured by comparing object dimensions derived from theimage data of the object with the unit scale. From this, volumemeasurements in litres for example, of a part of the object beingmonitored may be calculated. By accurately being able to takemeasurements of the object during monitoring, the differential data maybe generated with a high level of accuracy.

The calibration object may have three points of known relative positionand distance visible in the field of view of the recording system. Thecalibration object could, as an example, be in the form of a ‘wand’waved over a patient before the monitoring begins, or a known printed orprojected pattern of precise dimensions on the table, chair or bed onwhich the patient lies, which is used to brace the patient's back. Thecalibration object may of course in other embodiments be other objectsof known dimensions.

The unit calibration may be conducted only once, assuming that thephysical set up and positioning of the monitoring apparatus is notchanged. If the physical set up is changed, the unit calibration willneed to be conducted again.

For either, or both, the projector and recording system calibration, andthe unit calibration, the calibration pattern may be similar, or thesame, as the pattern of radiation described previously for themonitoring method. FIG. 7 shows an exemplary calibration patternprojected onto a saline bag. Further, in other embodiments, thecalibration pattern may include a sequence of different calibrationpatterns projected consecutively onto the calibration object, andrecording calibration image data for each of the different patterns. Forexample, the sequence of patterns may include a number of distinctpatterns, typically about sixteen for example, projected in rapidsequence onto the object. The forms of the patterns can be arranged toprovide successively finer pitched features within the patterns. Theprojected patterns of radiation may be, but are not restricted to, ageneric form of a simple pattern which is repeated (tessellated) in bothhorizontal and vertical (X and Y) directions. Further, in otherembodiments, for the projector and recording system calibration, thecalibration pattern may be interleaved with the pattern projected ontothe object during the monitoring method. In other embodiments, projectorand recording system calibration image data may be derived from thefirst and/or second image data; the projected pattern for objectmonitoring may include pattern characteristics for use in suchcalibration.

As shown in FIGS. 8A, 8B and 9, one or more patterns of known structure(including but not restricted to regular structures such as lines orpoints) may then be projected onto the object, after unit calibration,to start the monitoring of the object. These patterns may be fine orcoarse or a combination of both depending on requirements for systemsize, accuracy and fields of view.

The recording system 24 then captures images of the area(s) of theobject, as explained previously, and the patterns of structuredradiation as they appear on the object, capturing informationrepresentative of object configuration, for example, surface shape,movement and three dimensional configuration shape.

The above embodiments are to be understood as illustrative examples ofthe invention. Further embodiments of the invention are envisaged.

Further embodiments of the method are envisaged which include recordingfurther image data representing at least a portion of the projectedpattern of radiation on the object at a plurality of further instants intime, such as consecutive third and fourth instants in time. The furtherimage data may comprise a plurality of image data recorded successivelyat a plurality of time instants after the first and the second timeinstants, which represent a three dimensional configuration of theobject at the respective further instants in time. The further imagedata is processed to generate differential data representative of achange in the configuration of the object between at least two of theplurality of further instants in time. The properties of the furtherimage data may be similar to those of the first and second image datadescribed above. Accordingly, the further image data may be recorded,processed and/or used in a similar manner as any recording, processingand/or use of the first and/or second image data described previously.In this way, the method of the invention can monitor a change inconfiguration, such as a volume, of the object over a period of timespanning the first, second and further instants in time. Accordingly,the configuration change of an object may be monitored continuously.

In further exemplary embodiments, the monitoring apparatus describedabove may be used for animal monitoring and veterinary care, datacapture for body research, bio-modeling, and avatar animation, inaddition to other applications where monitoring of a three dimensionalconfiguration of an object is required.

Further, the method of the invention may monitor breathing and otherlife signs in, for example, a hospital, medical centre or local practiceenvironments or in mobile environments, for example,paramedic/ambulatory, field hospitals (civilian, military) or disasterrecovery centres.

The projection system may include a light source (e.g. a collimated LEDsource) illuminating a set of optical patterns created on a physicalmedium, for example film or glass, and sequenced via mechanical system,for example a rotated wheel on which the patterns are mounted and whichis positioned in front of the light source.

The angular field of view may be increased by using more than tworecording systems and more than one projecting system so that moresurface area of the object/subject may be captured and ensuring that thearea to be captured has at least two of the n cameras deployed. FIG. 6illustrates one configuration to enhance scanning coverage of thesubject/object.

As described above, a three dimensional representation of at least partof the object can be derived using the plurality of images captured atone time instant of the projected pattern on the object, using forexample two cameras. By taking a plurality of images for one timeinstant, image data with a greater coverage of the object, and/or withreduced occlusion problems of a certain object part, can be obtained. Inother advantageous embodiments of the invention, the recording systemmay include only one camera which records one image of at least aportion of the projected pattern on the object at the first and second,and possibly further, time instants. A three dimensional representationof at least part of the object may be derived for each time instant byinterpreting the recorded pattern in each respective image. Accordingly,in such embodiments, multiple cameras are unnecessary.

The apparatus may comprise the recording system and projection systemand the image data captured by the recording system may be provided in asuitable form for processing, to generate the image data representativeof the three dimensional configuration and the differential data. Suchprocessing of the image data may be processed remotely from therecording and projection systems and as such the processing system maybe located in another room or may be located in another country,connected to the recording and projection system via a computer network,and arranged to receive the image data from the recording and projectionsystem which are arranged to provide the image data to the processingsystem.

It is to be understood that any feature described in relation to any oneembodiment may be used alone, or in combination with other featuresdescribed, and may also be used in combination with one or more featuresof any other of the embodiments, or any combination of any other of theembodiments. Furthermore, equivalents and modifications not describedabove may also be employed without departing from the scope of theinvention, which is defined in the accompanying claims.

1. A method of monitoring an object, the method including: a) projectinga pattern of radiation onto an object for monitoring; b) recording at afirst instant in time first image data representing at least a portionof said projected pattern of radiation on the object, said first imagedata being representative of a three dimensional configuration of saidobject at said first instant in time; c) recording at a second instantin time second image data representing at least a portion of saidprojected pattern of radiation on the object, said second image databeing representative of a three dimensional configuration of said objectat said second instant in time; and d) processing said first and secondimage data to generate differential data representative of a change inthe configuration of said object between the first and second instantsin time.
 2. A method according to claim 1, wherein the method includesprocessing said first and second image data to generate differentialdata representative of a change in volume of said object between thefirst and second instants in time.
 3. A method according to claim 1,wherein the portions in steps b) and c) are substantially the sameportion of the pattern, or wherein the portions in step b) and c) aredifferent portions of the pattern.
 4. A method according to claim 3,wherein the method includes processing the first and second image datato identify gross movement of the object between the first and secondinstants in time.
 5. A method according to claim 1, wherein in step a) asequence of different patterns of radiation is projected onto theobject, each of the different patterns being projected consecutively ata different instant in time.
 6. A method according to claim 1,including: recording further image data representing at least a portionof the projected pattern of radiation on the object at a plurality offurther instants in time, said further image data being representativeof a three dimensional configuration of said object at each of theplurality of further instants in time; processing said further imagedata to generate differential data representative of a change in theconfiguration of said object between at least two of said plurality offurther instants in time.
 7. A method according to claim 6, wherein thefurther image data is recorded at a rate of 50 to 360 instants in timeper second.
 8. A method according to claim 1, wherein the methodincludes approximating the configuration of a portion of the object uponwhich the pattern of radiation is not projected.
 9. A method accordingto claim 8, wherein the method includes using the first and/or secondimage data for said approximating.
 10. A method according to claim 8,wherein the method includes generating the differential data using saidapproximated configuration of the portion of the object.
 11. A methodaccording to claim 1, wherein the object is a person.
 12. A methodaccording to claim 11, wherein the method includes processing said firstand second image data with bio-physical model data characterising theperson.
 13. A method according to claim 12, wherein the bio-physicalmodel data includes parameters representative of size, age, sex and/orbody type of the person.
 14. A method according to claim 11, wherein thechange of configuration is indicative of a lung function of the person.15. A method according to claim 1, wherein the method further includescalibrating of apparatus arranged to perform at least steps a) and b),said calibrating including: projecting a calibration pattern ofradiation onto a calibration object; and recording calibration imagedata representing at least a portion of said projected calibrationpattern of radiation, said calibration image data being representativeof a three dimensional configuration of said calibration object, andprocessing said first and second image data in step d) with thecalibration image data to generate the differential data.
 16. A methodaccording to claim 15, wherein the projecting in step a) furtherincludes projecting said calibration pattern, or at least one of saidcalibration patterns onto the object for monitoring, and the firstand/or second image data recorded in steps b) and/or c) includecalibration image data representative of characteristics of a projectionsystem and recording system of apparatus for performing said method. 17.A method according to claim 16, wherein the projected pattern ofradiation and the projected calibration pattern or at least one of thecalibration patterns are interleaved with each other.
 18. A method ofmonitoring an object, the method including: a) projecting a pattern ofradiation onto an object for monitoring; b) recording at a first instantin time first image data representing at least a portion of saidprojected pattern of radiation on the object, said first image databeing representative of a three dimensional configuration of said objectat said first instant in time; c) recording at a second instant in timesecond image data representing at least a portion of said projectedpattern of radiation on the object, said second image data beingrepresentative of a three dimensional configuration of said object atsaid second instant in time; and d) providing said first and secondimage data to a processing system for generating differential datarepresentative of a change in the configuration of said object betweenthe first and second instants in time.
 19. A method of monitoring anobject, the method including: a) receiving first and second image datarecorded by the method of: i) projecting a pattern of radiation onto anobject for monitoring; ii) recording at a first instant in time thefirst image data which represents at least a portion of said projectedpattern of radiation on the object, said first image data beingrepresentative of a three dimensional configuration of said object atsaid first instant in time; and iii) recording at a second instant intime the second image data representing at least a portion of saidprojected pattern of radiation on the object, said second image databeing representative of a three dimensional configuration of said objectat said second instant in time, and b) processing said first and secondimage data to generate differential data representative of a change inthe configuration of said object between the first and second instantsin time.
 20. A method of monitoring an object, the method including: a)at a first instant in time, projecting a pattern of radiation onto anobject for monitoring; b) recording at the first instant in time firstimage data representing at least a portion of said projected pattern ofradiation on the object, said first image data being representative of athree dimensional configuration of said object at said first instant intime; c) at a second instant in time, projecting a different pattern ofradiation onto the object for monitoring; d) recording at the secondinstant in time second image data representing at least a portion ofsaid projected different pattern of radiation on the object, said secondimage data being representative of a three dimensional configuration ofsaid object at said second instant in time; and e) processing said firstand second image data to generate differential data representative of achange in the configuration of said object between the first and secondinstants in time.
 21. An apparatus for monitoring an object, theapparatus comprising: a projection system arranged to project a patternof radiation onto an object for monitoring; a recording system arrangedto record at first and second instants in time, respectively, first andsecond image data representing at least a portion of said projectedpattern on the object, said first and second image data beingrepresentative of a three dimensional configuration of said object atsaid first and second instants in time, respectively; and a processingsystem arranged to process said first and second image data to generatedifferential data representative of a change in the configuration ofsaid object between the first and second instants in time.
 22. Anapparatus as claimed in claim 21, wherein the recording system isarranged to record a plurality of images of at least a portion of theprojected pattern of radiation at one instant in time to record imagedata representative of a three dimensional configuration of the object,each of the plurality of images being recorded at a different viewpointof said object.
 23. An apparatus according to claim 22 comprising apattern generator arranged to generate said pattern of radiation forprojection by the projection system.
 24. An apparatus according to claim23, wherein the pattern generator is arranged to generate at least onecalibration pattern for projection onto the object for monitoring and/ora calibration object.
 25. Apparatus comprising: a projection systemarranged to project a pattern of radiation onto an object at a firstinstant in time and at a second instant in time; and a recording systemarranged to record at the first instant in time first image datarepresenting at least a portion of the pattern projected on the objectat the first instant in time and to record at the second instant in timesecond image data representing at least a portion of the patternprojected on the object at the second instant in time, said first andsecond image data being representative of a three dimensionalconfiguration of said object at said first and second instants in time,respectively, wherein the apparatus is operable to provide said firstand second image data to a processing system operable to process saidfirst and second image data to generate differential data representativeof a change in the configuration of said object between the first andsecond instants in time.
 26. Apparatus comprising: a processing systemarranged to process first and second image data to generate differentialdata representative of a change in the configuration of an objectbetween a first and a second instant in time, wherein the apparatus isoperable to receive said first and second image data from furtherapparatus comprising: a projection system arranged to project a patternof radiation onto the object at the first instant in time and at thesecond instant in time; and a recording system arranged to record at thefirst instant in time the first image data which represents at least aportion of the pattern projected on the object at the first instant intime and to record at the second instant in time the second image datawhich represents at least a portion of the pattern projected on theobject at the second instant in time, said first and second image databeing representative of a three dimensional configuration of said objectat said first and second instants in time, respectively.
 27. A computerprogram product comprising a non-transitory computer-readable storagemedium having computer readable instructions stored thereon, thecomputer readable instructions being executable by a computerised deviceto cause the computerised device to perform the method of claim
 1. 28. Acomputer program product comprising a non-transitory computer-readablestorage medium having computer readable instructions stored thereon, thecomputer readable instructions being executable by a computerised deviceto cause the computerised device to perform the method of claim
 18. 29.A computer program product comprising a non-transitory computer-readablestorage medium having computer readable instructions stored thereon, thecomputer readable instructions being executable by a computerised deviceto cause the computerised device to perform the method of claim 19.